
Macrothink 

Institute™ 


Journal of Educational Issues 

ISSN 2377-2263 
2015, Vol. 1, No. 2 


A Smart Way of Coping with Common Core Challenges 
- Introduction to CAFA SmartWorkbook 

Jaehwa Choi (Corresponding author) 

Assessment, Testing, and Measurement 
Department of Educational Leadership, The George Washington University 
2134 G Street NW #106, Washington, DC 20052, USA 
Tel: 1-202-944-2602 E-mail: jaechoi@gwu.edu 

Miseon Kang 

Assessment, Testing, and Measurement, The George Washington University 
2134 G Street NW #106, Washington, DC 20052, USA 
E-mail: avecmiseon@gwu.edu 

Najung Kim 

Department of Educational Leadership, The George Washington University 
2134 G Street NW #106, Washington, DC 20052, USA 
E-mail: najungkim609@gmail.com 

William Dardick 

Assessment, Testing, and Measurement, The George Washington University 
2134 G Street NW #105, Washington, DC 20052, USA 
E-mail: wdardick@gwu.edu 

Xinxin Zhang 
University of Alberta 

116 St. and 85 Ave., Edmonton, AB, Canada 
E-mail: xinxin4@ualberta.ca 


70 


www.macrothink.org/jei 



.U Macrothink 
All Institute™ 


Journal of Educational Issues 

ISSN 2377-2263 
2015, Vol. 1, No. 2 


Received: October 1, 2015 Accepted: November 5, 2015 
Published: November 12, 2015 

doi:10.5296/jei.vli2.8381 URL: http://dx.doi.org/10.5296/jei.vli2.8381 


Abstract 

The Common Core State Standards (CCSS) in mathematics are currently adopted in most U.S. 
states. Nonetheless, most math teachers across the country are still experiencing difficulties in 
putting these standards into practice. Teachers and local school administrators are faced with 
a challenge of adapting methodologies in instruction and assessment to ensure that students 
master the knowledge and skills required in the new standards. This leads to an urgent need 
for well-designed teaching and assessment tools for math education that are aligned to the 
CCSS. 

The purpose of this paper is to illustrate the Computer Adaptive Formative Assessment 
(CAFA) SmartWorkbook which is an Information and Communication Technology (ICT) 
based teaching and assessment tool specially designed for coping with challenges in 
implementing the CCSS in mathematics. The CAFA SmartWorkbook represents a new stage 
in exploring opportunities in educational innovation, capitalizing on advances in assessment 
and technology. This system can be an effective solution to cope with CCSS challenges in 
both theoretical and practical points of view for students, teachers, parents, and educational 
administrators. 

Keywords: Common Core State Standards, Formative assessment, Computer adaptive testing, 
Computer assisted learning, Auto item generation, Computer Adaptive Formative Assessment, 
Math education 

1. Introduction 

Nearly every state in the U.S. has begun implementing the Common Core State Standards 
(CCSS; Williams, 2012) in mathematics, while many math teachers are still unsure how to 
effectively put these standards into practice. Teachers and local school administrators need to 
make fundamental changes in how they teach math in order for students to master the 
knowledge and skills required in the new standards. 

There is an urgent need for well-designed teaching tools in math that are aligned to the CCSS 
(Komhaber, Griffith, & Tyler, 2014). For example, there is a need at middle school level for 
tools in mathematics driven by the expectations of what a student can do. Students are 
expected to compute with fluency, work with algebraic expressions, solve real-world 
geometry problems, build an understanding of statistics and probability, and learn other 
challenging math content in the CCSS. A 2013 survey of more than 400 middle school math 
teachers by the University of Rochester found that slightly over two-thirds of the teachers 
surveyed are using textbooks adopted prior to the CCSS (Davis, Choppin, Roth, & Drake, 
2013). The study further states that one-third of the teachers cited a need for more support, 
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including better-aligned curriculum materials, in implementing the math CCSS. In the 
absence of these materials, over 60% of the teachers surveyed reported regularly accessing 
online resources, and others are creating their own curriculum materials. Creating classroom 
assignments for middle school math aligned to the CCSS is a demanding and time-consuming 
task, more so when considering the complexity of knowledge and skills embodied in the 
standards for those grades. 


In recent years there has been a general consensus on a number of challenges associated with 
CCSS. First, there has been an unsuccessful campaign in articulating to parents, teachers and 
the general public what the students are expected to learn under the CCSS. Primarily this 
challenge can be tied to a lack of sufficient information being provided to these critical 
stakeholders. Second, CCSS are not fully aligned to textbooks, digital media and curricula to 
meet the international standards. Third, there is a gap in available resources for professional 
development of educators on identified needs and best practices of CCSS. Lastly, an 
assessment system to measure student performance against the standards is not fully ready to 
be implemented in practice. This makes assessment data inaccurate which leads to challenges 
in related policy changes (Williams, 2012; Shanahan, 2013). 

The assessments currently being designed by state consortia and others to measure students’ 
mastery of CCSS are primarily summative, in that they are intended to evaluate students’ 
overall proficiency for accountability purposes or to quantify how much students have 
learned during the year. While summative assessment is certainly useful, it often leaves a 
teacher scrambling for specific content to help students with different needs. These 
assessments are unlikely to provide the kind of value added information of formative 
assessment, which are integrated into classroom practice. Teachers need formative 
assessment in order to pinpoint students’ challenges in mastering specific standards and to 
individualize instruction to help each student overcome particular difficulties. It is imperative 
to have a genuine formative approach which inherently adapts the contents with which the 
students are struggling to simultaneously review the student’s capabilities and provide timely 
feedback/treatment for strengthening their weak areas. 


The CAFA Smart Workbook (Choi, Kim, & Yoon, 2014b), an ICT based assessment tool 
specially designed for coping with challenges in implementing the CCSS in mathematics, has 
recently been developed based on an ICT Assessment Engineering platform called the 
Computer Adaptive Formative Assessment (CAFA; Choi, Kim, & Yoon, 2012) system. The 
application prototype currently includes an inventory of math problems aligned to one cluster 
of the math standards in the Common Core. 


The primary application of this ICT-based assessment tool is to help fill the need for 
well-designed classroom assignments with items aligned to the CCSS in middle school math. 
The CAFA SmartWorkbook can be used as a teaching resource. For example, a teacher can 
select assignments that are fully aligned to each of the standards in the Common Core math 
for grades 6 and 7. The assignments are designed to help teachers effectively teach math 
using methods from the CCSS. The assignments can be used as homework, practice problems, 
quizzes and other classroom assessments. 
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The application has several desirable features not available in many of the existing 
CCSS-aligned curriculum materials. First, it provides students with several types of built-in 
support for learning. After students submit an answer to a problem, they can receive 
immediate feedback on their answers, with detailed steps toward a solution and tips about 
how to solve the problem. The system can provide additional sets of practice items geared to 
students’ level of knowledge, as well as an automatic reassessment for each standard to help 
students retain what they have just learned or previously mastered. Second, SmartWorkbook 
can automatically adapt the difficulty level of problems to an individual student’s needs and 
progress, using a diagnostic and treatment algorithm built into the CAFA system. This makes 
it possible for teachers to tailor assignments to an individual student’s learning needs. Third, 
the application provides primary stakeholders (e.g. students, parents, and teachers) with 
instantaneous diagnostic reports about the difficulties students are experiencing in learning. 
Feedback is instantaneous, enabling teachers to diagnose students’ daily progress and identify 
problem areas tailored to the child in learning specific knowledge and skills in the Common 
Core math. Fourth, it is flexible to use. It is not necessary for every student to have an 
individual laptop or desktop to use SmartWorkbook. The assignments can be used online 
through the web or off-line in a traditional paper format. In addition, students and their 
parents can access the application with a smart phone or tablets, using the type of Quick 
Response (QR; Denso Wave Incorporated, n.d.) barcodes that have become commonplace in 
advertising and shopping. Because the application can be easily accessed on demand 
regardless of time and location via any mobile devices, it offers a ubiquitous source of 
curriculum material, assessment items, and diagnostic information. 

This paper introduces the CAFA SmartWorkbook; a Math workbook, specially developed to 
cope with the Common Core challenges using state-of-the-art technologies by experts in 
areas of Educational Assessment, Computer Science, and Math. Our paper is organized into 
four main sections. First we briefly review the background of the CCSS and challenges of 
implementing the CCSS in practice. Next we provide a general overview of formative 
assessment systems. We then introduce the CAFA system and CAFA SmartWorkbook. Next 
we review technical components that illustrate benefits of the CAFA system and CAFA 
SmartWorkbook. We close this manuscript with a discussion of the implications of the system 
and future work from three different perspectives: Educational Policy, Educational Practice, 
and Educational Technology. 

2. Common Core State Standards 

2.1 Common Core State Standards 

According to the Common Core State Standards Initiative (2015) “The Common Core is a set 
of high-quality academic standards in mathematics and English language arts/literacy (ELA). 
These learning goals outline what a student should know and be able to perform at the end of 
each grade. The standards were created to ensure that “all students graduate from high school 
with the skills and knowledge necessary to succeed in college, career, and life, regardless of 
where they live”. The main purpose of CCSS is to identify the most essential skills and 
knowledge that students should have, not the way in which they acquire them (Rothman, 
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2011). Under the CCSS, students will be held to the same standards, which is one of the 
essential issues for educational equity. The standards provide common goals in terms of 
increased skill levels that students will need in order to be competitive in the 21 st century 
world. The implementation of the CCSS is a state-led effort. The creation of the standards 
brings public educators from across the nation together to share the expertise and common 
values, which are so important for citizen participation in a democratic society (Williams, 
2012; Shanahan, 2013; South Pasadena Unified School District, 2014-2015). 

2.2 Challenges in CCSS 

Since its inception, many challenges have been identified in the implementation of CCSS 
(Greene, 2012; Toscano, 2013). There was a disconnect in communicating to parents, 
teachers, and the general public what students are expected to learn. Parents and teachers are 
still experiencing difficulties in understanding the value of CCSS and how to use the CCSS 
primarily due to a lack of stakeholder information and resources for teachers. CCSS is not yet 
fully aligned to textbooks, digital media and curricula to meet the international standards. 
Many textbooks are only partially aligned to CCSS. There is a need for professional 
development on identified needs and best practices related to CCSS. Without this resource, 
teachers are not able to fully understand and incorporate the new standards into their teaching. 
There is also needs for a working assessment system that could evaluate student performance 
against the standards. This misalignment leads to inaccuracy in assessment data which 
hinders policy changes. 

3. Assessment Systems 

3.1 Forms of Assessment 

Formative assessment is a compilation of observed tasks and behaviors designed to promote 
student learning by altering educational activities through a feedback mechanism between 
teacher/student (Carnegie Mellon University, 2015). Effective assessment of student 
performance in math as well as in other subjects can be as important as teaching standards. 
Assessment provides teachers and policy makers with important information about student 
achievements and gaps in learning (Boston, 2002). Along with the positive features of 
assessment there are limitations of the usefulness of acquired information. Despite the efforts 
of many teachers to develop tests that assess critical thinking and understanding of the 
material, some tests are still designed to encourage superficial learning. Tests can encourage 
competition among students rather than promote individual improvement. Assessments often 
fulfill administrative requirements rather than identify specific needs of the student 
(Birenbaum et al., 2006; Black & William, 1998a; Black & William, 1998b). 

There are different forms of assessment beyond the formative model of student teacher 
feedback. When we think of the “traditional assessment” we are likely considering the 
evaluation that occurs at the end of learning module that is intended to encapsulate the 
summation of student work and compare it to an external criterion (2015). These high stakes 
assessments are summative assessments, while low stakes assessments are often formative in 
nature. High stakes summative assessment can occur in the classroom setting or be more 
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formalized in standardized tests, such as those required by legislation (i.e. No Child Left 
Behind [NCLB], 2002). Traditional assessment based on legislation is driven by curricula 
content. States determine the knowledge and skills that a student must learn in each grade, 
and then the student must complete an assessment designed to measure his or her 
understanding of this knowledge and skill set (Mueller, n.d.). These assessments have been 
criticized for covering a very limited range of content at a very superficial level (Senk, 
Beckman, & Thompson, 1997). It also should be noted that when assessments are presented 
in the classroom setting it may be difficult or impossible to parse out the goal of assessment 
simply from the structure of the test. 


There exists tension between classroom-based formative assessments of student learning, and 
the highly visible and large-scale national or regional summative assessments of student 
performance. The latter of which are often intended to hold schools accountable for meeting 
standards, and may hold particular consequences for low or underperforming schools. Highly 
visible summative tests are often used to hold schools accountable for student achievement 
and drive what happens in classrooms. There is disconnection between the value of 
information gathered by the large-scale summative approach and formative classroom-based 
approach to assessment and evaluation. Information gathered through national regional or 
school-based monitoring is seen as irrelevant or unhelpful to the business of teaching. 

Similarly, information gathered in classrooms is seen as irrelevant to the business of policy 
making (Tiknas & Sutton, 2006). According to a study on Assessment in Education by Tiknas 
and Sutton (2006) teachers lack a clear understanding of the differences between formative 
and summative assessment. They further attribute this relatively weak understanding as one 
of the challenges teachers face when they try to develop formative assessment tasks for their 
classrooms. In considering the requirement and positive aspects of formative assessment, 
coupled with challenges in the classroom, we introduce a system that can help support the 
teacher and provide valuable formative feedback. 


4. Computer Adaptive Formative Assessment (CAFA) System 

4.1 Inception 

With regards to the impact of feedback in computer-based learning system (van der Kleij, 
2014), the results that are reviewed in literature are rather mixed. A number of authors have 
provided positive impact on students’ learning outcomes as a consequence of particular ways 
to provide feedback (Corbalan, Paas, & Cuypers, 2010; Lee, Lim, & Grabowski, 2010; Smits, 
Boon, Sluijsmans, & van Gog, 2008; Wang, 2011). While other studies have found no or little 
impact (Clariana & Lee, 2001; Gordijn & Nijhof, 2002; Kopp, Stark, & Fischer, 2008). 

Recently, to enable the system-based formative assessment for learning, researchers have 
developed several ICT applications, such as EAGLE (Enhanced Assessments of Grade-Level 
Expectations; Louisiana Department of Education, n.d.), ALEKS (Assessment and Learning 
in Knowledge Space; McGraw-Hill Global Education Holdings, 2014), or PowerSource 
(Phelan, Vendlinski, Choi, Dai, Herman, & Baker, 2011). Since many challenges have been 
identified for schools, parents, and students in coping with CCSS, researchers (Choi, Kim, & 
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Yoon, 2012) developed the CAFA system, a technology-mediated educational learning and 
assessment system focused on a formative assessment process. The CAFA system is designed 
to provide individually tailored diagnostics, feedback, and other useful information to 
students, educators, parents, policy makers, and members of the community. 

4.2 Structure 

CAFA system is composed of a set of important components. First, CAFA system is strictly 
standard-based which means it is completely aligned with the Common Core standards. 
Second, it is ICT-based for educational big data analytic system, i.e., all assessment items are 
electronically managed in the network-based central database system using client-server 
structure (Farlex Incorporation, 2015). Third, it is built based on an assessment engineering 
framework equipped with Automatic Item Generation (AIG; Gierl & Haladyna, 2012; Choi, 
Kim, & Yoon, 2014a) system. Therefore, the system can provide nearly a limitless set of 
assessment items. This assessment system also provides highly interactive and multi-media 
assessment items using several multi-media technologies, such as parameterized mathematical 
expression image generation, parameterized chart/figure generation, or text-to-speech. Lastly, 
CAFA incorporates an accountability system where performance and progress made by 
students and teachers are closely monitored. Clients of CAFA can use diverse platforms to 
access CAFA server without having to download all the information - students or teachers can 
use websites, mobile applications or eBbooks in order to access the server to check out the 
information they need. Using the information saved on the server, users can create their own 
version of SmartWorkbook or SmartAssignment which are two examples of client applications 
using the CAFA system as an engine. They can also work online while connected to the server 
or print out the materials and work offline using QR (Denso Wave Incorporated, n.d.) codes for 
instant feedback. 

4.3 Objectives 

Considering the challenges mentioned previously surrounding CCSS and its assessment, 
CAFA was developed to serve the following objectives. First, it will help articulate to parents, 
teachers and the general public what the students are expected to learn. Second, CAFA will 
provide aligned textbooks, digital media and curricula that meet the international standards via 
a workbook solution, which will be very useful unlike many existing textbooks claim to be 
aligned with CCSS but in truth present no linking evidence between CCSS and those textbooks. 
Third, it will also provide a base for professional development for educators on identified needs 
and best practices via a teacher friendly application and help to implement an assessment 
system to measure student performance against the Standards via formative assessment system. 
Lastly, it will help evaluate policy changes needed to help students and educators meet the 
Standards via research based service. 

5. CAFA SmartWorkbook 

5 .1 Background 

Like other client application using the CAFA engine, such as CAFA SmartAssignment (Choi, 
Kim, & Yoon, 2013; Kim, H., Park, J., Jung, Y., Park, S., Kim, C., Lee, C., & Choi, J., 2014), 
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the CAFA SmartWorkbook is also an application using the CAFA engine (or server). The 
CAFA SmartWorkbook was created (a) in response to challenges identified with CCSS; (b) to 
maximize the full benefits of CCSS; (c) to be aligned textbooks; and (d) to be aligned of the 
other assessment systems. The workbook contributes greatly to broadened access to Common 
Core curriculum by providing the materials and learning tools to users through smart devices. 

5.2 Development 

The CAFA SmartWorkbook was developed by Associate Professor Jaehwa Choi at the George 
Washington University along with his team members; Professor Sunhee Kim from Howard 
Community College’s mathematics division; and Professor Kyongil Yoon from Notre Dame of 
Maryland University’s computer studies department. The SmartWokrbook is a math workbook 
which supports both traditional and smart-learning environments for K-12 students in the 
United States. This interdisciplinary and inter-institute project is expected to contribute to the 
enhancement of mathematics educational services especially for implementing CCSS. 
Designed specifically to cope with challenges in implementing CCSS, the workbook is 
currently available for students, teachers and for 6 th grade curriculum. This free and printable 
workbook, fully aligned with CCSS, integrates smart-learning components. These components 
can be accessed via any mobile and/or smart devices, and offers instant feedback including 
detailed solution steps, additional practices on problem areas, as well as diagnostic information. 
By virtue of these integrations, this workbook will play an extremely important role not only 
in online education, but also in offline classroom educational practices. 

5.3 Advantages 

First, SmartWorkbook provides students with several types of built-in support for learning. 
After students submit an answer to a problem, they can receive immediate feedback on their 
answer, as well as the detailed steps toward a solution or hints about how to solve the 
problem. The system can provide additional sets of practice items geared to students’ level of 
knowledge, as well as an automatic reassessment for each standard to help students retain 
what they have just learned or previously mastered. Second, SmartWorkbook can 
automatically adapt the difficulty level of problems to an individual student’s needs and 
progress, using a diagnostic and treatment algorithm built into the CAFA system. This makes 
it possible for teachers to tailor assignments to an individual student’s learning needs. Third, 
the application provides students, parents and teachers with instantaneous diagnostic reports 
about students’ difficulties in learning. This feature is particularly useful because the 
feedback arrives instantaneously and enables teachers to diagnose students’ daily progress, as 
well as their problem areas in learning specific knowledge and skills in the math Common 
Core. Fourth, from the workbook’s versatile nature, it is not necessary for every student to 
have an individual laptop or desktop to use it. The assignments can be used online through 
the web or offline in a traditional paper format. In addition, students and their parents can 
access the application with a smart phone - which most people now have access to—or 
tablets, using the type of QR barcodes that have become commonplace in advertising and 
shopping. As the application can be easily accessed on demand regardless of time and 
location via any mobile devices, it offers a ubiquitous source of curriculum material, 
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assessment items, and diagnostic information. This year, Professor Choi’s team will expand 
this service from grades 6 and 7 to include grades 3 to 8 with capability of incorporating a 
multi-lingual version including other languages (e.g., Chinese, Spanish, and Korean) for 
students to be prepared and positioned to compete successfully in the global economy. 


5.4 A Solution for CCSS Challenges 

This application helps fill the need for well-designed classroom assignments and assessment 
items aligned to the CCSS in middle school math. This application includes assignments that 
are fully aligned to each standard in the Common Core math for grades 6 and 7 (grade 
expansion in process) and are developed to help teachers effectively teach math using new 
methodologies proposed in the CCSS. These assignments can be used as homework, practice 
problems, quizzes and other classroom assessments. In particular, because the CAFA 
SmartWorkbook is designed to overcome the challenges of CCSS in practices, it can be a 
useful resource to help teachers, students, and parents who are facing difficulties in applying 
CCSS. Five major benefits of the workbook are herein illustrated in more detail. 

First, the CAFA SmartWorkbook is fully aligned to CCSS. Unlike other math assessment 
materials and workbooks that are partially (e.g., only about 30 percent) aligned to CCSS, this 
workbook strictly and fully covers all CCSS and all assessment items are carefully developed 
to be aligned to each standard. Inside of the workbook, the practice page contains at least four 
practice problems which are specifically designed for a particular standard. Furthermore, for 
grades 6 and 7, all standards were covered in the workbook. 

Second, by virtue of being fully aligned to CCSS, this workbook helps to articulate to parents, 
teachers and the general public what the students are expected to learn. Each standard 
practice page in the workbook contains a description of each standard, practice problems, 
results, and assessment feedback is organized, referenced, and linked by standard information. 
These features of the workbook will help parents and teachers better understand the 
information and/or value of the CCSS and how to use and benefit from it. 


Third, this workbook can be a base for professional development for educators on identified 
needs and best practices. By virtue of covering all standards with fully functioning ICT 
features, this workbook can be a great tool for understanding CCSS. Therefore, this 
workbook can help teachers struggling to use CCSS as the base of professional development 
where they lack resources. Such resources, which are crucial in understanding the new 
standards and being able to utilize and incorporate them into their teaching, are either not yet 
developed or not available to all teachers. In these cases the workbook can act as such a 
resource. 


Forth, this workbook system can be extended toward an assessment system to measure 
student performance against the standards. Educational activities within this workbook, such 
as answering on a practice test, can be turned into and saved in the server within the ICT 
environment as part of the assessment information. Using such saved information, it is 
possible to inform students’ learning progress on particular standards. This year, 
approximately 12 million students will be assessed against the new standards by large groups 
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and associations that are delegated to carry out the assessment. However, they are not fully 
ready to implement formative assessment due to gaps between the technological 
advancements and the practical assessment system in use. In sum, this workbook can improve 
the accuracy of assessment data via the synergetic integration of ICT and psychometric 
techniques in a workbook application which also has much broader impact capability. 

Lastly, this workbook can provide valuable information to evaluate policy changes needed to 
help students and educators meet the standards. Once collected, saved, and analyzed inside of 
a server in ICT environment, such assessment information can be used as an efficient tool to 
monitor and evaluate students’ progress and achievement on the standards. Policy makers and 
evaluators can use the results of such analysis to evaluate and develop policies to improve 
students’ learning. 

6. Features of the SmartWorkbook 

We now introduce and demonstrate various useful features of the SmartWorkbook. Most 
importantly, individuals can use SmartWorkbook to practice a specific standard with various 
practice questions, take an exam, submit answers, and receive a report card with a diagnosis 
immediately after submitting their answers (Choi, Kang, & Kim, 2014). 

6.1 Getting Started 

As shown in Figure 1, by accessing the website at http://SmartWorkbook.CAFALab.com, 
users can create their own workbook by selecting grade, domain and cluster level. Upon 
receiving the workbook through email, users can choose to work online or have the 
workbook printed to work offline. 
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10:38 AM 


Grad# h 

4.EL.A 


Smart Workbook ™ 

Jm. r . il l i J flHTATA 



CAFA SmartWorkbook™ 



Select a Grade/Domain/Cluster 


Grade: 

Not Selected 


Domain: 

All domains 


Cluster: 

All clusters 


E-Mail: 

| sfutures@yahoo.com 



Send a Workbook to My eMail 

Copyright O 2014 eMathTest Inc. All rights reserved. 


Figure 1. CAFA SmartWorkbook: Creating a new account 

Note. Reprinted from CAFA SmartWorkbook, 2014, Retrieved from February 23, 2015 from 
http://SmartWorkbook.CAFALab.com/. Copyright 2015 by CAFA Lab Incorporation. 
Reprinted with permission. 


6.2 Practice Page 

As shown in Figure 2, the first page that a user will see, when opens the workbook, will be 
this Practice Page which contains practice problems for a certain standard. Practice 
problems’ numbers and figures that are used in the questions have been generated according 
to the specifications that the user has provided. And on the top right comer of each item, 
users will see a QR code. 
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CAFA SmartWorkbook 1 

CCSS: 6.EE.1.010 
Description: 


Difficulty: Easy 


Correct %: N/A 


what is equivalent expression using whole 
ber exponents for the expression, 
5x5+3x3x3+4x47 


CD 3 5 + 3 s + 2 4 
®8 12 


Figure 2. CAFA SmartWorkbook: Accessing practice page, scanning a QR code to access 

item page 

Note. Reprinted from CAFA SmartWorkbook, 2014, Retrieved from February 23, 2015 from 
http://SmartWorkbook.CAFALab.com/. Copyright 2015 by CAFA Lab Incorporation. 
Reprinted with permission. 


6.3 Item Practice Page 

As shown in Figure 2, by using a smart device, users can scan each item’s QR code (Denso 
Wave Incorporated, n.d.) which will take them to a specific item page. In this page, users can 
access various additional information such as the common core standard information for the 
item, item difficulty, item correct %, etc. 
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CAFA SmartWorkbook ™ 


CC SS: 6 EE.1 QIO 
Description: 

Difficulty: Easy 

C&rreCt %\ NyA 


What is equivalent expression using whole 
number exponents for the expression, 
2x2x213x3x3+4x4? 

® 2 3 + 3 3 + e © 3 * + 4 2 

<T>S B 

(?) 3 3 + 3 3 ■+ 2 J 




© 




CAfA Lab i 
Ccpyri^ihl r 



CCSS: 6.EE. 1.01 CAFA SmartWorkbook ™ 

Description: _ 

Difficulty: Easy CCSS: 6.EE.1.010 _ 

Correct %: N/A Description: 


“Difficulty: Easy 



CAFA Lab | About SmanWorkbook I Terms I Privacy 

Copyright © 2014 eMathTest, Inc. All rights reserved. 


Figure 3. CAFA SmartWorkbook: Checking answers and practicing variants 

Note. Reprinted from CAFA SmartWorkbook, 2014, Retrieved from February 23, 2015 from 
http://SmartWorkbook.CAFALab.com/. Copyright 2015 by CAFA Lab Incorporation. 
Reprinted with permission. 


6.4 Checking Answers 

At the bottom of each item box, as shown in Figure 3, users will see five different icons. By 
clicking the second icon which resembles a light bulb, users can see Correct Answer and 
Solution Steps for the item. All items contain up to ten solution steps that illustrate step by 
step process of solving the problem. 

6.5 Practicing Variants and Other Items 

As shown in Figure 3, the two arrows icon demonstrates one of the paramount features of the 
SmartWorkbook. By clicking this button, the system will generate problems with the same 
pattern but with different numbers and figures which will help students understand and 
master the specific item. Note that this particular workbook feature was strongly appreciated 
and endorsed in practice. An empirical experiment research conducted in a South Korea high 
school as a pilot-test of the workbook application and its features. In this pilot test, the 
students and teachers feedback in the focus group analysis result showed that this feature was 
the most important and attractive feature of the workbook (H. Kim, J. Park, Jung, S. Park, C. 
Kim, Lee, & Choi, 2014). By clicking the one arrow icon, users can access other items that 
also fall under the same standard to further master the standard. 

6.6 Asking Questions 

By clicking the question mark button from the menu bar, students can send questions to 
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system operators. Once the button has been clicked, a question box will pop up at the bottom 
of the page. Students can then write their question in the box and click “send question” button 
to send the inquiries to the system operators. 


* t'^.d 19:37 PM 


Ask a Question 


Item Information 

What square unit Is the area of die shaded 
triangle below? 
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1. The area of a triangle is the half of the 
length rimes the width. 

So, the area of the shaded triangle i3 

1 

— x 4 :< 4 = 8 square unit. 

Your Email: 

Subject: Question about item9072201 
Question: 

write your question 


Figure 4. CAFA SmartWorkbook: Asking questions 

Note. Reprinted from CAFA SmartWorkbook, 2014, Retrieved from February 23, 2015 from 
http://SmartWorkbook.CAFALab.com/. Copyright 2015 by CAFA Lab Incorporation. 
Reprinted with permission. 


6 .7 Taking a Test 

The working application also provides assessment function. Once students feel ready to take 
a test on a cluster or domain of CCSS, they can take a test in an exam mode page. After they 
solve problems, by scanning the exam page QR code with a smart device, users will be able 
to access the page where they can submit answers. 
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Diagnosis Report: 6.EE.A 
User: sfutures@yahoo.com 

Overall Proficiency: 13.33 % 



Skill Proficiency (6.EE. 1.010): 0.00 % (0/1) 
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Skill Proficiency (6.EE.1 02b): 0 00 % (0/1) 


Figure 5. CAFA Smart Workbook: Taking a test and receiving report cards 

Note. Reprinted from CAFA SmartWorkbook, 2014, Retrieved from February 23, 2015 from 
http://SmartWorkbook.CAFALab.com/. Copyright 2015 by CAFA Lab Incorporation. 
Reprinted with permission. 


6.7 Checking Answers & Report Card 

Once students have submitted their answers, they can immediately check their final score. By 
clicking on each item number, they can access a page where they can find more similar 
questions. By clicking the “Show Report” button, users can instantly see their scores, unlike 
other traditional reports that usually take a few weeks to a few months. The report card is 
provided with various types of diagnostic information prepared based on students’ 
performance. 

7. Discussion 

The purpose of this paper was to introduce the CAFA SmartWorkbook, a Math workbook, 
tailor-made to cope with the Common Core challenges by using state-of-the-art technologies 
designed by experts in areas of Educational Assessment, Computer Science, and Math. In the 
previous parts of this paper, we reviewed the background of the CCSS and the challenges of 
implementing the CCSS in practice. We also examined how the CAFA SmartWorkbook can 
be an effective solution to cope with such challenges in both theoretical and practical points 
of view for students, teachers, parents, and educational administrators. We also reviewed 
technical components of this system focusing on identifying which specific technologies have 
been used to bring forth the benefits of the workbook. 

Educational assessment is a crucial part of the quest for educational innovation and 
improvement. By appropriately analyzing educational assessment information, stakeholders 
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in education are able to monitor how well students are achieving and whether students and 
institutions are progressing toward the educational goals that have been set, e.g., 
implementing CCSS. We have noted the lack of resources for educational stakeholders in 
implementing CCSS. Additionally, the goals to inform and improve education through 
assessment are limited by current educational practices. 


Educational assessment, the technological infrastructure and monitoring of achievement have 
dramatically changed over the last century. It is clear that such advances interact with the 
applied challenges in educational practice regarding CCSS. Although monitoring has been a 
crucial component of evaluating the educational system, advances in ICT and futuristic 
assessment can provide information beyond the current practices. 

There is a great deal of information regarding the future of assessment, in terms of its 
elements, rules, and guidelines that can be used to develop and use them. But there remains 
the need for knowledge and skills to be further analyzed to provide clear guidance for an 
assessment framework. For example, further domain analysis is necessary for the CAFA 
Smart Workbook to incorporate subjects other than math. The CAFA Smart Workbook 
components should be further developed to be connected to large-scale accountability 
systems and to be linked to other learning applications beyond the workbook scope, such as 
textbook, game-based learning, muli-media learning, etc. 


It would be highly valuable if educational stakeholders could come to an agreement on the 
current assessment system (including tools, resources and assessment) information, even 
though it may have some limitations. Integrating the advances in educational assessment and 
technology will provide more informed benchmarks for valid and supportive assessment. It is 
important to provide students and teachers with practical and efficient tools. Further we 
should recognize the need to fairly and effectively measure students’ learning information 
from multi-sources, assembled to high level standards such as those described in the CCSS. 

Evidence drawn from formative assessment (such as learning profiling information on 
multiple components of achievement level) is more useful and meaningful in practice when 
compared to summative assessment results such as overall performance scores. Future 
challenges the CAFA SmartWorkbook intends to consider include various aspects of 
knowledge and competence above and beyond overall test scores or performance data. For 
example, the future system could be an integrated framework to analyze (a) how students 
understand and explain concepts; (b) how to solve problems; (c) how to measure state/level 
of knowing; and (d) how to enable self-paced learning. 


We have demonstrated throughout the illustrations of the CAFA SmartWorkbook achieving 
goals such as those mentioned requires coherent integration of both educational assessments 
and ICT field. Without such incorporation, we may continue to struggle with the challenges 
in practice because the results of mismatched analysis may be at best, partially correct, or at 
worst deceptive or misleading. Whether it be formative or summative orientation of 
assessments, it is imperative to have sufficient and valid information about what has been 
learned in order to move toward appropriate next steps for learning progress. 
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Given the advancements in education and technology ICT enhanced assessment tools and 
systems should not just be an option, but rather should be the leading paradigm for future 
development. The futuristic assessment design should support multidisciplinary and 
collaborative environment. For example, the system should support all stakeholders (e.g., 
teachers, administrators, cognitive scientists, subject matter specialists, and psychometricians) 
so that they can participate collaboratively in the entire assessment process, e.g., from the 
design process to the final reporting stage. 

Finally we considered policy and practice for professional development needed to 
appropriately operate the new system and to identify how assessment should be 
developed/used to maximize the quality of educational services. Substantial endeavors are 
warranted to implement such improvements in educational assessment. Undoubtedly, it is 
clear that such commitment is crucial for our next generation of teachers as it is evident that 
the new system will enhance student learning. 
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